<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
    <title>ShellCheck: SC2094 – Make sure not to read and write the same file in the same pipeline.</title>
    <link rel="stylesheet" href="css/bootstrap.min.css" />
  </head>
  <body style="margin-left: auto; margin-right: auto; max-width: 800px">
    <h1>SC2094 – ShellCheck Wiki</h1>
    <a href="https://github.com/koalaman/shellcheck/wiki/SC2094">See this page on GitHub</a>
    <p style="display: none"><a href="index.html">Sitemap</a></p>
    <hr />
    <h2
id="make-sure-not-to-read-and-write-the-same-file-in-the-same-pipeline">Make
sure not to read and write the same file in the same pipeline.</h2>
<h3 id="problematic-code">Problematic code:</h3>
<div class="sourceCode" id="cb1"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb1-1"><a href="SC2094.html#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="fu">grep</span> foo file.txt <span class="kw">|</span> <span class="fu">sed</span> <span class="at">-e</span> <span class="st">&#39;s/foo/bar/g&#39;</span> <span class="op">&gt;</span> file.txt</span></code></pre></div>
<h3 id="correct-code">Correct code:</h3>
<div class="sourceCode" id="cb2"><pre class="sourceCode sh"><code class="sourceCode bash"><span id="cb2-1"><a href="SC2094.html#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="fu">grep</span> foo file.txt  <span class="kw">|</span> <span class="fu">sed</span> <span class="at">-e</span> <span class="st">&#39;s/foo/bar/g&#39;</span> <span class="op">&gt;</span> tmpfile <span class="kw">&amp;&amp;</span> <span class="fu">mv</span> tmpfile file.txt</span></code></pre></div>
<h3 id="rationale">Rationale:</h3>
<p>Each step in a pipeline runs in parallel.</p>
<p>In this case, <code>grep foo file.txt</code> will immediately try to
read <code>file.txt</code> while <code>sed .. &gt; file.txt</code> will
immediately try to truncate it.</p>
<p>This is a race condition, and results in the file being partially or
(far more likely) entirely truncated.</p>
<p>Note that this can also be a problem when you write to a file and
read from it later in the pipe. The second command (which reads the
file) may not see all the output of the first. An exception in this case
is a non-greedy file reader like <code>less</code>, for example
<code>python foo.py 2&gt; errfile.txt | less - errfile.txt</code> will
successfully allow you to see stdout and stderr separately in less.</p>
<h3 id="exceptions">Exceptions</h3>
<p>You can ignore this error if:</p>
<ul>
<li>The file is a device or named pipe. These files don't truncate in
the same way.</li>
<li>The command mentions the filename but doesn't read/write it, such as
<code>echo log.txt &gt; log.txt</code>.</li>
</ul>
<h3 id="additional-resources">Additional Resources</h3>
<ul>
<li><a
href="https://stackoverflow.com/questions/6696842/bash-redirect-input-from-file-back-into-same-file">StackOverflow:
bash redirect input from file back into same file</a></li>
<li><a
href="https://mywiki.wooledge.org/BashPitfalls#pf13">BashPitfalls:
<code>cat file | sed s/foo/bar/ &gt; file</code></a></li>
</ul>
    <hr />
    <p style='font-size: 80%'><a href="../index.html">ShellCheck</a> is a static analysis tool for shell scripts. This page is part of its documentation.</p>
  </body>
</html>


